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LONG-TERM  GOALS 

This  project  seeks  to  develop  the  technology  to  fuse  high  spatial  resolution  MultiSpectral  Imagery 
(MSI)  with  lower  spatial  resolution,  but  higher  spectral  resolution,  HyperSpectral  Imagery  (HSI)  to 
provided  enhanced  target  detection  and  battlespace  characterization.  It  seeks  to  push  beyond 
traditional  PAN  sharpening  techniques  to  develop  applications  that  will  increase  the  accuracy  and 
fidelity  of  the  sharpened  spectral  imagery. 

OBJECTIVES 

1)  Co-register  high  resolution  MSI  data  with  HSI  data  collected  during  the  Fall  of  2004. 

2)  Correct  the  MSI  data  for  illumination  and  atmospheric  interference  to  derive  a  broadband  remote 
sensing  reflectance,  Rrs,  measurement  from  the  MSI  data. 

3)  Develop  methodologies  to  fuse  the  HSI  data  with  the  higher  spatial  resolution  MSI  data,  using  the 
spectral  characterization  of  the  MSI  Rrs  data  stream. 

APPROACH 

The  fusion  of  remote  sensing  data  from  different  sensors  has  a  long  history  of  success  in  the  terrestrial 
environment  (Pohl  and  Genderen,  1998).  Its  application  in  ocean  remote  sensing  has  accelerated  in 
recent  years  with  the  use  of  multiple  resolution,  multiple  frequency  optical  and  microwave  imaging. 
The  optical  imagers  include  both  active  (e.g.  LIDAR)  and  passive  (e.g.  HyperSpectral  Imagers,  HSI). 
Microwave  imagers  are  typically  Synthetic  Aperture  Radar  (SAR).  While  the  field  has  been  generally 
confined  to  airborne  and  space-based  platforms,  imagery  data  fusion  may  also  include  some 
combination  of  data  from  these  platforms  with  data  from  in-water  systems  such  as  multibeam  and  side- 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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scan  sonar  imagery.  In  short,  data  fusion  is  the  process  by  which  two  or  more  streams  of  data  of 
different  temporal  and  spatial  resolution  are  combined  to  produce  synthesized  products  which  are 
unattainable  from  a  discrete  image. 

The  advantages  of  data  fusion  are  fairly  clear.  In  the  temporal  domain,  data  fusion  allows  for  the  more 
complete  imaging  of  an  area  intermittently  covered  by  clouds.  It  may  also  provide  better  analysis  of 
movement  or  time-dependent  change  in  targets  and/or  environmental  conditions.  In  the 
electromagnetic  frequency  domain,  targets  which  are  visible  in  one  image  type  may  be  less  than  visible 
in  another,  and  vice  versa.  SAR  imagery  may  illuminate  target  elevations  or  directional  wave  spectra, 
whereas  optical  imagery  may  identify  targets  by  shape  and  color.  Combining  the  targeting  and 
identification  of  these  different  data  types  provides  target  recognition  and  environmental 
characterization  with  high  confidence  and  lower  false  alarm  rates  than  may  be  found  from  single 
source  imagery.  In  the  spatial  domain,  objects  that  are  identifiable  on  the  basis  of  size  and  shape  in 
panchromatic  (PAN)  imagery  are  often  unidentifiable  by  their  geometric  dimensions  in  lower 
resolution  imagery  multispectral  or  hyperspectral  imagery.  The  converse  is  that  the  spectral  imagery 
provides  the  ability  to  identify  targets  or  characterize  the  environment  based  on  unique  spectral 
signatures,  which  is  unavailable  in  the  panchromatic  imagery,  but  may  not  be  able  to  resolve  the  shape 
or  dimensions  of  small  targets. 

This  project  seeks  to  co-locate  simultaneously  collected  MSI/HSI  data  streams  to  develop  enhance 
imagery  sharpening  capabilities  for  use  in  target  detection  and  environmental  characterization.  It  seeks 
to  build  upon  PAN  sharpening  techniques  by  using  the  full  spectral  information  provided  by  MSI  data 
stream.  We  hypothesize  that  the  best  way  to  attempt  this  image  fusion  is  at  the  level  of  remote  sensing 
reflectance,  so  the  MSI  data  must  be  corrected  for  illumination  and  atmospheric  interference.  The 
spectral  information  from  the  co-registered  MSI/HSI  Rrs  data  will  then  be  fused  to  provide  more 
accurately  sharpened  HSI  data  for  spectral  target  algorithms. 

WORK  COMPLETED 

This  project  began  in  April  of  2005,  and  started  with  data  previously  collected  by  FERI.  FERI,  in 
collaboration  with  the  California  State  University  System,  under  the  NOAA-funded  California  Center 
for  Integrative  Coastal  Observation,  Research  and  Education  (CICORE)  Program 
(http://www.feriweb.org/projects/ci-core/)  recently  (Fall  2004)  collected  over  5,000  square  kilometers 
of  0.3  meter  MSI  and  3  meter  HSI  data  over  a  wide  variety  of  coastal  environments  along  the 
California  coast  (http://www.feriweb.org/projects/ci-core/arcims/).  The  survey  sites  include  Humbolt 
Bay,  San  Francisco  Bay,  Monterey  Bay,  the  Big  Sur  coast,  San  Luis  Bay,  Santa  Barbara,  Newport,  and 
San  Diego  Harbor.  The  MSI  instrumentation  was  provided  under  a  cooperative  agreement  with  the 
Applanix  Systems  Integration  Group  (ASIG)  subsidiary  of  the  Trimble  Corporation.  This  MSI  system 
(Digital  Sensor  System;  http://www.applanix.com/products/dss_index.php)  was  provided  as  part  of  the 
Applanix  Position  and  Orientation  System  (POS  AV; 

http://www.applanix.com/products/posav_index.php),  which  was  being  evaluated  as  a  replacement  for 
the  current  POS  system  on  the  NRL  PHILLS  2  airborne  hyperspectral  imager. 

The  raw  DSS  imagery  (MSI  imagery)  and  POS  data  was  provided  to  FERI  by  ASIG  after  the  flights. 
The  processing  and  geo-positioning  of  the  MSI  data  was  completed  by  FERI  under  this  ONR  project. 
This  geo-positioning  system  is  of  much  greater  accuracy  than  the  previous  POS  system  on  the  PHILLS 
2.  The  processing  of  this  POS  data  stream  required  additional  training,  which  was  provided  by  ASIG. 
In  addition,  the  DSS  is  a  frame  camera  imagery,  which  is  very  different  from  the  line  scanning  imager, 
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which  is  at  the  heart  of  the  PHILLS  2.  The  processing  of  the  MSI  data  stream  itself  required  additional 
training  in  the  photogrammetry  and  ortho-rectification.  The  training  was  completed,  as  well  as  the 
processing  of  the  raw  POS  data  and  DSS  imagery  to  ortho-rectified  (Figure  1).  These  data  are  to  be 
used  with  the  co-registered  HSI  data  from  the  Morro  Bay  region  (Figure  2). 


Figure  1.  FERI  MSI  of  selected  region  in  Morro  Bay,  CA  collected  November  11,  2004.  This  area 
has  been  chosen  as  a  focus  site  for  the  development  of  the  MSI/HSI  fusion  algorithms.  Note  the 
enhanced  spatial  resolution  when  compared  to  the  HSI  imagery  in  Figure  2. 


RESULTS 

The  MSI/HSI  data  has  been  co-registered  in  anticipation  of  the  next  step  in  the  development  of  the 
spectral  fusion  over  the  next  6  months.  One  issue  that  has  been  found  during  the  training  of  FERI 
personnel  by  ASIG,  which  may  be  cause  for  difficulties,  is  that  the  radiometric  and  spectral 
calibrations  of  the  DSS  are  not  what  have  been  purported  by  the  manufacturer.  The  calibration  issues 
need  to  be  resolved  prior  to  the  development  of  the  Rrs  fusion  approach,  for  it  will  be  very  difficult  to 
correct  for  illumination  and  atmospheric  effects  without  accurate  calibration.  This  recalibration  of  the 
MSI  sensor  by  FERI  personnel  is  scheduled  to  occur  this  fall  (funded  by  Trimble),  so  these  data  should 
be  available  later  this  year  to  complete  the  initial  development  of  the  Rrs  fusion  approach. 
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IMPACTS/APPLICATIONS 


The  abilities  to  detect  targets  and  characterize  the  environment  in  the  Very  Shallow  Water  (VSW)  and 
beach  zone  are  critical  to  MIW,  MCM,  and  NSW  operations.  The  image  fusion  techniques  here  will 
provide  enhanced  abilities  to  detect  and  characterize  target  in  access  denied  areas  using  both 
geomorphology  (shape)  and  spectral  (color)  signatures.  In  addition,  the  successful  development  of 
these  techniques  will  feed  back  into  the  development  of  new  imaging  systems  for  space  and  aircraft 
platforms,  including  Unmanned  Aircraft  Systems  (UAS). 


Figure  2.  FERI HSI  of  selected  region  in  Morro  Bay,  CA  collected  November  11,  2004.  This  area 
has  been  chosen  as  a  focus  site  for  the  development  of  the  MSI/HSI  fusion  algorithms.  Note  the 
reduced  spatial  resolution  when  compared  to  the  MSI  imagery  in  Figure  1. 

RELATED  PROJECTS 

This  project  works  closely  with  those  of  Pis  Bissett  and  Kohler,  including  N000 1401 10201, 

N0001403 10626, and N000140410297 
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